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(54) Method and apparatus for image improvement 



(57) A method and an apparatus for improving the 
image quality. The method comprises steps of produc- 
ing image correcting information to decrease errors in 
the image to be produced by a camera module (109, 
1 1 1 , 1 1 2) by comparing the image taken by said camera 
module (109, 111, 112, 113) with a test image, storing 
the image correcting information produced and identify- 
ing information related to said image correcting informa- 
tion in the memory of an electronic peripheral device 



(1 04, 106), receiving the image produced by said cam- 
era module (1 09, 111, 112, 113) and a second piece of 
identifying information related to said image in the elec- 
tronic peripheral device (104, 106), comparing said 
identifying information with said second piece of identi- 
fying information In said peripheral device, and perform- 
ing an image improvement operation on said image in 
said peripheral device in response to the comparison 
carried out. 
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Description 

[0001] The present invention relates to utilization of 
camera properties in image processing. The invention 
especially, but not necessarily, relates to improvement 
of the quality of an image produced by a digital camera. 
[0002] I n digital cameras and digital video cameras an 
optical image is converted into an electronic format by 
an image sensor, which is typically a photosensitive 
semiconductor element (Charge Coupled Device CCD 
or Complementary Metal oxide Semiconductor CMOS). 
This element is a plate known as a detector matrix which 
consists of small and regular photosensitive and colour- 
sensitive picture elements, i.e. pixels. The resolution of 
the matrix varies according to its physical size and fine- 
ness. An image sensor typically comprises hundreds of 
thousands of pixels, e.g. 640 x 480 = 307 200 pixels are 
used in VGA resolution (video graphics array). Utiliza- 
tion of advanced CMOS technology in the production of 
image sensors has enabled integration of digital and an- 
alogue electronics and the image sensor into the same 
semiconductor element. The size and weight of the 
camera module have been reduced, which has made it 
possible to integrate it into smaller and smaller electron- 
ic devices, such as mobile imaging phones. 
[0003] The quality of an image produced by a camera 
module is usually proportioned to the display properties 
of the communication device, e.g. a mobile imaging 
phone. On a small screen the size of the picture ele- 
ments that create an image, i.e. pixels, is smaller than 
in monitors used in offices, and thus at a certain resolu- 
tion the sharpness of the image appears to be better to 
the human eye. The image quality on the screen of the 
communication device can be improved e.g. by provid- 
ing the image with a greater contrast and more saturated 
colours. This makes the image seen on the screen of 
the communication device appear more natural to the 
viewer and improves its visibility. 
[0004] When an image produced by a mobile imaging 
phone is transferred e.g. onto a computer screen which 
has better display properties than the mobile imaging 
phone, errors will appear in the image on the computer 
screen, such as discoloration or distortions. Due to this 
the image quality may have to be adjusted or improved. 
Furthermore, since the computing capacity of comput- 
ers is nowadays considerably greater, a better image 
quality can be achieved e.g. using a computer program 
intended for image improvement. However, the user 
may find computer program of this kind difficult to use, 
and thus the user must have basic knowledge about the 
function and properties of image processing software to 
achieve the desired result in image improvement. Fur- 
thermore, the image improvement operation has to be 
performed separately on each image, which the user 
may find troublesome and time consuming. 
[0005] The present invention provides a method and 
an apparatus for improving the image quality and espe- 
cially, but not necessarily, for improving the quality of an 



image produced in a mobile imaging phone in a periph- 
eral device, such as a computer or a server. Properties 
resulting from the errors of the camera module used for 
taking images are stored as image correcting informa- 
5 tion either in the memory of the camera module or in the 
memory of the electronic device into which the camera 
module is integrated or to which it is otherwise connect- 
ed. Alternatively, the correcting information can be 
stored in the memory of a peripheral device, such as a 
10 computer or a server, which is capable of processing the 
image information produced by the camera module. 
[0006] The image correcting information of the cam- 
era module is stored preferably during the manufacture 
of the camera module or the electronic device, such as 
15 a mobile imaging phone. The errors the camera module 
has caused in the image it has produced, such as optical 
distortions of the camera module lens, discoloration, 
and problems resulting from the quality of the camera 
module, such as noise and focus errors, can be reduced 
by comparing the image taken by the camera module 
and a test image. One or more image correcting param- 
eters obtained as a result of the comparison are stored 
as image correcting information. The parameters are 
preferably stored in the memory of the server, but the 
image correcting information can also be stored in the 
memory of the camera module, mobile imaging phone 
or computer, for example. Storing of the image correct- 
ing information during the manufacturing process im- 
proves the usability of the device, for example. In that 
case image conversion does not require information on 
the properties of the device that has taken the image 
(camera module, mobile imaging phone) because this 
information has been stored in the device in advance. 
Thus image processing can be automated further and 
made faster. Transmission of the image in the original 
format e.g. from a mobile imaging phone reduces the 
amount of capacity required from the mobile imaging de- 
vice because image processing is not carried out until 
in a peripheral device, such as a computer or a server. 
[0007] A first aspect of the invention relates to a meth- 
od of improving the quality of an image produced by a 
camera module in an electronic peripheral device, which 
is capable of processing said image, characterized in 
that the method comprises: storing image correcting in- 
formation related to said camera module in the memory 
of the electronic peripheral device, receiving the image 
produced by said camera module and identifying infor- 
mation related to said image correcting information in 
said electronic peripheral device, and performing an im- 
age improvement operation on said image in said pe- 
ripheral device using the image correcting information 
to which said received identifying information relates. 
[0008] A second aspect of the invention provides an 
electronic peripheral device for improving the quality of 
an image produced by a camera module in said periph- 
eral device, which is capable of processing said image, 
characterized in that said peripheral device comprises: 
a memory for storing image correcting information relat- 
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ed to said camera module in the memory of said periph- 
eral device, receiving means for receiving the image 
produced by said camera module and identifying infor- 
mation related to said camera module, and image im- 
provement means for improving the quality of said im- 
age using in image improvement the image correcting 
information to which said received identifying informa- 
tion relates. 

[0009] A third aspect of the invention provides a sys- 
tem for improving the quality of an image produced by 
a camera module in an electronic peripheral device, the 
system comprising at least one camera module which 
can produce an image and at least one electronic device 
which can transmit said image and identifying informa- 
tion related to the image to an electronic peripheral de- 
vice, characterized in that the system further comprises: 
a memory for storing the image correcting information 
related to said camera module for said peripheral de- 
vice, means for transmitting the image produced by said 
camera module and the identifying information related 
to said image correcting information to a peripheral de- 
vice, and image improvement means for performing an 
image improvement operation to improve the quality of 
said image using the image correcting information to 
which said identifying information relates. 
[0010] A fourth aspect of the invention provides a 
computer program product for improving the quality of 
an image produced by a camera module in an electronic 
peripheral device, which is capable of processing said 
image, characterized in that the computer program 
product comprises: computer program means for mak- 
ing said peripheral device store the image correcting in- 
formation related to said camera module in the memory 
of said peripheral device, computer program means for 
making said peripheral receive the image produced by 
said camera module and identifying information related 
to said image correcting information, and computer pro- 
gram means for making said peripheral device perform 
image improvement to improve the quality of said image 
using the image correcting information to which said re- 
ceived identifying information relates. 
[001 1] In the following, the invention will be described 
in greater detail with reference to the accompanying 
drawings, in which 

Figure 1 illustrates a system according to an em- 
bodiment of the invention, 

Figure 2 is a flow chart showing a method according 
to an embodiment of the invention, 

Figure 3 illustrates a camera module according to 
an embodiment of the invention, 

Figure 4 illustrates a communication device accord- 
ing to an embodiment of the invention, and 

Figure 5 illustrates a peripheral device according to 



an embodiment of the invention. 

[0012] Figure 1 illustrates a system according to an 
embodiment of the invention, the system comprising 
5 one or more mobile imaging phones 101, 102, which 
comprise a camera module 111,112. The system further 
comprises a server 106 and a communication network 
103. An image produced by the mobile imaging phone 
101, 1 02 can be transmitted over the network 1 03 to a 
*o server 1 06, which can store the image in a storage me- 
dium 108, or print the image by a printer 107. The sys- 
tem may also comprise e.g. a communication device 
1 1 0, which can communicate over the network 1 03, and 
a computer 1 04, which can receive the image produced 
by the mobile imaging device 1 01 , 1 02 e.g. over the net- 
work 103 and e.g. print the image by a printer 105. A 
camera module (references 1 09, 1 1 3) can be connected 
to the communication device 110 and the computer 1 04 
for transmitting the image produced over the communi- 
cation network 1 03 to a server 1 06, for example. 
[0013] The image correcting information of the cam- 
era module 1 09, 1 1 3 is stored in the memory of the serv- 
er 106 preferably during the manufacture of the camera 
module. The image correcting information of the camera 
module 113 can also be stored in the memory of the 
computer 104, for example, and the image correcting 
information of the camera module 1 09 in the memory of 
the communication device 110, for example. The image 
correcting information of the camera module 1 11 , 1 1 2 is 
stored in the memory 108 of the server 106 preferably 
during the manufacture of the camera module orthe mo- 
bile imaging phone 101, 102. The image correcting in- 
formation of the camera module 111, 112 can also be 
stored in the memory of the mobile imaging phone 101 , 
1 02 or the computer 1 04. 

[0014] The mobile imaging phone 102 can receive 
and transmit images produced by the camera module 
to another mobile imaging phone 101, computer 1 04, or 
server 106 e.g. over a Bluetooth connection or a com- 
munication network 103, such as a mobile communica- 
tion network or the Internet. The image produced by the 
camera module 112 of the mobile imaging phone 102 
can be transmitted to the server 1 06, for example. The 
Image to be transmitted is also provided with an identi- 
fier by means of which the server 1 06 can identify the 
mobile imaging phone 112 that has transmitted the im- 
age and perform an image improvement operation on 
the image received using the image correcting informa- 
tion that corresponds to the identifying information and 
is stored in the memory 108. The server 106 can store 
the improved image in the memory 1 08, or if the user of 
the mobile imaging device 112 so wishes, transmit the 
image to a printing service for printing by a printer 107. 
The printing service can be implemented e.g. as follows: 
the user of the mobile imaging phone 112 informs the 
server 1 06 of his/her name and address when transmit- 
ting an image, after which the printed image can be 
mailed to the user of the mobile imaging phone 112, for 
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example. The user of the mobile imaging phone 102 
can, if he/she so wishes, retrieve the improved image 
from the memory 1 08 for the computer 1 04 later e.g. via 
the Internet network and print the image by the printer 
105. 

[001 5] The camera module 1 09 can be connected to 
the communication device 110 e.g. by a cable, a Blue- 
tooth connection or an optical connection, such as an 
infrared connection. The image produced by the camera 
module 109 can be transmitted e.g. by the communica- 
tion device 110 over the communication network 103 to 
the server 1 06. The identifier of the camera module 1 09 
is also transmitted at the same time so that the server 
106 can perform an image improvement operation on 
the basis of the image correcting information related to 
the identifier of the camera module 109. 
[0016] The camera module 113 can be connected to 
the computer e.g. by a cable, a Bluetooth connection or 
an optical connection, such as an infrared connection. 
The image produced by the camera module 113 can be 
transmitted e.g. by the computer 104 over the commu- 
nication network 1 03 to the server 1 06. The identifier of 
the camera module 1 13 is also transmitted at the same 
time so that the server 1 06 can perform an image im- 
provement operation on the basis of the image correct- 
ing information related to the identifier of the camera 
module 113. Alternatively, the image correcting informa- 
tion corresponding to the identifier of the camera module 
113 can be retrieved for the computer 1 04, and thus the 
computer 1 04 can perform an image improvement op- 
eration on the image produced by the camera module 
113. 

[001 7] The computer 1 04 may be a portable computer 
or a workstation and it is capable of receiving images 
produced by the mobile imaging phone 1 02 either via a 
communication network 103, e.g. the Internet, or on the 
Bluetooth connection or on a similar wireless connec- 
tion, or over a cable connecting the devices. From the 
image information the computer 1 04 receives it can sep- 
arate camera information which is associated with im- 
age information in the mobile imaging phone. The com- 
puter 1 04 can process the images to be displayed on 
the basis of the above-mentioned camera information. 
A printer 105 can also be connected to the computer 
104 for printing the images. The computer can display 
the processed images by means of a monitor or the 
above-mentioned printer. 

[001 8] Figure 2 shows a method according to an em- 
bodiment of the invention. In step 201 image correcting 
information of the camera module is produced, prefera- 
bly during the manufacture of the camera module. If the 
camera module is integrated into an electronic device 
such as a mobile imaging phone, the image correcting 
information can be produced during the manufacture of 
said mobile imaging phone. Alternatively, said image 
correcting information can be produced e.g. when the 
camera module or the mobile imaging phone is used for 
the first time. 



[0019] The image correcting information is produced 
by comparing the image taken by the camera module 
with a test image, and image correcting parameters can 
be generated on the basis of this comparison according 

5 to the enor type. The correcting information may include 
optical errors or image correcting parameters for cor- 
recting errors resulting from the quality of the camera 
module. Optical errors include faults the camera module 
lens causes in the image and colours. Errors resulting 

io from the quality of the camera module include fixed pat- 
tern noise caused by the pixel structure, i.e. thermal 
noise, which increases as the amount of light decreas- 
es. Focus errors, i.e. crispening errors, occur particular- 
ly in connection with lenses with a fixed focus, e.g. the 

15 whole image is not focused. In that case the centre of 
the image, for example, can be sharp whereas the edg- 
es are fuzzy. Noise and sharpness are geometrical dis- 
tortions that change radially from the centre of the op- 
tics, in which case the image correcting parameters are 

20 expressed as the centre of distortion and elliptic form for 
each colour component R, G and B (red, green and 
blue). The darkness of the image also changes radially 
towards the image edges. Darkness can be eliminated 
e.g. using a function of the 6 th order having the form 1 

25 +ar 2 +- br 4 + cr 6 , where r is the distance from the centre 
and constants a, b and c are numbers that are defined 
on the basis of the test image. 

[0020] In step 202 the correcting information is stored 
preferably in the memory of the peripheral device, such 

30 as a server, a mobile imaging phone or a computer, but 
it can also be stored in the memory of the camera mod- 
ule. The identifying information of the camera module is 
stored at the same time. Alternatively, the correcting in- 
formation can be transferred from the memory of the 

35 camera module, mobile imaging phone or computer into 
the memory of the server e.g. by transmitting the cor- 
recting information and identifying information of the 
camera module, mobile imaging phone or computer to 
the server. The server stores the correcting information 

to and identifying information and links these data with 
each other. The server can perform an image improve- 
ment operation on the image information it has received 
on the basis of the correcting information stored in ad- 
vance and the identifier it has received. 

45 [0021] In step 203 an image is produced in the camera 
module or mobile imaging phone. The image can be a 
still image or a video image which can be stored e.g. in 
the memory of said mobile imaging phone. 
[0022] In step 204 the identifying information of the 

50 camera module or the mobile imaging phone and the 
image information are transmitted to a peripheral de- 

formation can alternatively be inserted into the above- 
mentioned image information in the mobile imaging 
55 phone, for example. 

[0023] In step 205 the image information and correct- 
ing information are received by a peripheral device, 
such as a server or a computer, and the image improve- 
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ment operation is performed on the basis of the correct- 
ing information. 

[0024] Figure 3 illustrates a camera module 300 ac- 
cording to an embodiment of the invention. The camera 
module comprises a memory or a similar dataslot 301 
for the image correcting information, one or more optical 
lenses 302, a photosensitive CMOS element or a CCD 
sensor element 303, a control unit 304 for controlling the 
sensor element, a programmable analogue amplifier 
305 (Programmable Gain Amplifier PGA). An analogue 
image signal is converted into a digital format by an AD 
converter 306, after which it can be processed (refer- 
ence 307) by different image processing operations e. 
g. by adjusting the colour balance and/or the white bal- 
ance. Via the connection 308 the image signal is trans- 
mitted e.g. into the memory of the electronic device, 
such as the mobile imaging phone, or for display on the 
screen of the mobile imaging phone. 
[0025] The correcting information is stored in the 
memory 301 of the camera module, preferably during 
its manufacture. The memory 301 also comprises the 
identifier of the camera module, by means of which the 
peripheral device can identify the camera module and 
perform an image improvement operation on the image 
information corresponding to the identifying information. 
[0026] Figure 4 illustrates a communication device 
400 according to an embodiment of the invention, the 
communication device comprising a camera module 
300 for producing an image onto a screen 406 or into a 
memory 404, a transceiver 402 and an antenna 408 for 
transmitting and receiving data, e.g. image information, 
wirelessly, at least one application 405 for carrying out 
operations of the communication device, a processor 
403 and a memory 404 for performing operations of the 
communication device 400 and the application 405, a 
keyboard 407 for feeding commands into the communi- 
cation device 400. In addition, the communication de- 
vice 400 comprises image correcting information stored 
in the memory or in a similar storage medium 409. The 
memory 404 may further comprise the identifier of the 
communication device, by means of which the periph- 
eral device can identify the image information transmit- 
ted by the communication device and associate the im- 
age information with the corresponding image correct- 
ing information. 

[0027] The image correcting information is stored in 
the memory 409 preferably during the manufacture of 
the communication device or it can be stored in a stor- 
age medium, such as a CD, a DVD, a floppy disk or the 
like, in advance in some other manner. Alternatively, the 
correcting information can be produced afterwards, e.g. 
when the communication device is used for the first time. 
The camera module of the communication device 400 
is used for producing image information which is com- 
pared with the test image. The comparison can be per- 
formed and the image correcting information produced 
e.g. by a computer or a similar device. The result of the 
comparison is used for producing image correcting in- 



formation, which comprises at least one parameter that 
describes the interference in the image. 
[0028] The image information produced by the com- 
munication device is transmitted together with the image 

5 correcting information to a peripheral device, such as a 
server or a computer, which performs an image im- 
provement operation on said image information on the 
basis of said image correcting information. If the image 
correcting information is already stored in the peripheral 

io device, the identifying information of the communication 
device can be optionally transmitted with the image in- 
formation. 

[0029] Figure 5 illustrates a peripheral device 500 ac- 
cording to an embodiment of the invention. The periph- 

15 eral device is preferably a server, but it may also be a 
computer The peripheral device 500 comprises a stor- 
age medium 501 for storing image correcting informa- 
tion, a processor 502 and a memory 504 for controlling 
the operations of the peripheral device, a network inter- 

20 face 507 for receiving the image information and the cor- 
recting information and for transmitting corrected image 
information. The peripheral device 500 also includes a 
storage interface for storing the received correcting in- 
formation and the image information in a mass memory, 

25 such as a hard disk, and a printing interface 509 for print- 
ing the corrected image information received by a print- 
er. The peripheral device 500 may also comprise a key- 
board 506 for feeding commands into the peripheral de- 
vice, a monitor 503 for displaying the image information 

30 in visual form and at least one application 505 e.g. for 
performing an image improvement operation on the re- 
ceived image information on the basis of the image cor- 
recting information that corresponds to the image infor- 
mation and is already stored in the memory 501 of the 

35 peripheral device. 

[0030] The peripheral device 500 receives the image 
correcting information of the camera module, which is 
preferably produced already during the manufacture of 
the camera module or an electronic device, such as a 

40 mobile imaging phone, into which the camera module is 
integrated. At the same time the peripheral device 500 
also receives identifying information related to said cam- 
era module, electronic device or both. The identifying 
information can also be a serial number or another sim- 

45 jjar identifying code. 

[0031] The identifying information- and image correct- 
ing information received are stored in the memory 501 
of the peripheral device and linked with each other. 
When the peripheral device receives an image pro- 

50 duced by the camera module, it also receives identifying 
information, which is preferably related to the camera 
module, for example. The peripheral device compares 
the identifying information it has received with the iden- 
tifying information stored in the memory 501, and if the 

55 identifying information received is found in the memory 
501, the peripheral device performs an image improve- 
ment operation on the image it has received based on 
the image correcting information to which said identify- 



INSDOCJD: <EP 1 274224 A2 I > 



9 



EP 1 274 224 A2 



10 



ing information received and stored in the memory 501 
relates. 

[0032] The implementation and embodiments of the 
invention were described by means of examples above. 
It is obvious to a person skilled in the art that the inven- 5 
tion is not limited to the details of the embodiments de- 
scribed above and that the invention can be implement- 
ed otherwise without deviating from the characteristics 
of the invention. The embodiments shown should be re- 
garded as illustrative only, i.e. the embodiments and ap- 10 
plications of the invention are limited only by the ap- 
pended claims. Consequently, different optional embod- 
iments of the invention defined in the claims, including 
equivalent embodiments, fall within the scope of the in- 
vention. 15 



Claims 

1 . A method of improving the quality of an image pro- 20 
duced by a camera module in an electronic periph- 
eral device, which is capable of processing said im- 
age, characterized in that the method comprises: 

storing image correcting information related to 25 
said camera module in the memory of the elec- 
tronic peripheral device, 

receiving the image produced by said camera 
module and identifying information related to 30 
said image correcting information in said elec- 
tronic peripheral device, 

performing an image improvement operation 
on said image in said peripheral device using 35 
the image correcting information to which said 
received identifying information relates. 

2. A method according to claim 1 , characterized in 
that said image correcting information is produced 40 
by comparing the image taken by said camera mod- 
ule with a test image. 



6. A method according to claim 4, characterized in 
that said image correcting information and said 
identifying information are produced during the 
manufacture of the electronic device into which said 
camera module is integrated. 

7. A method according to claim 1 , characterized in 
that said image correcting information is also stored 
in at least one of the following: memory of said cam- 
era module, memory of the electronic device into 
which the camera module is integrated. 

8. A method according to claim 7, characterized in 
that said electronic device is a mobile imaging 
phone. 

9. An electronic peripheral device (500) for improving 
the quality of an image produced by a camera mod- 
ule (300) in said peripheral device (500), which is 
configured to process said image, characterized in 
that the peripheral device comprises: 

a memory (501 ) for storing the image correcting 
information related to said camera module in 
the memory of said peripheral device, 

receiving means (502, 507) for receiving the im- 
age produced by said camera module (300) 
and identifying information related to said cam- 
era module, 

image improvement means (501 , 502, 505) for 
improving the quality of said image using in im- 
age improvement the image correcting infor- 
mation to which said received identifying infor- 
mation relates. 

10. A device according to claim 9, characterized in 
that said receiving means (502, 507) are arranged 
to receive the image correcting information pro- 
duced during the manufacture of said camera mod- 
ule (300) and identifying information. 

11. A device according to claim 10, characterized in 
that said identifying information is the identifying in- 
formation of said camera module (300). 

12. A device according to claim 9, characterized in 
that said receiving means (502, 507) are arranged 
to receive the image correcting information pro- 
duced during the manufacture of the electronic de- 

/ a r\r\\ ~ ~~~~ i. .1 -~ /oaav :~ 
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connected, and the identifying information. 

13. A device according to claim 9, characterized in 
that said electronic device (400) is one of the fol- 
lowing: a mobile imaging phone or a mobile phone. 



3. A method according to claim 2, characterized in 
that said correcting information comprises at least *5 
one correcting parameter which describes the dif- 
ference between said image produced and said test 
image. 

4. A method according to claim 3, characterized in 50 
that said difference results from at least one of the 

following: optica! distortion, noise or darkening. 

5. A method according to claim 4, characterized in 
that said image correcting information and said 55 
identifying information are produced during the 
manufacture of said camera module. 
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14. A device according to claim 9, characterized in 
that said peripheral device (500) is one of the fol- 
lowing: a server, a computer. 

1 5. A system for improving the quality of an image pro- 5 
duced by a camera module (109, 111 , 112, 113) in 

an electronic peripheral device (104, 106), the sys- 
tem comprising at least one camera module (109, 
111,112,113), which is capable of producing an im- 
age, and at least one electronic device (101 , 102, 10 
110), which is capable of transmitting said image 
and identifying information related to the image to 
the electronic peripheral device (1 04, 1 06), charac- 
terized in that the system also comprises: 

15 

a memory (1 08) for storing the image correcting 
information related to said camera module for 
said peripheral device, 



image improvement means (104, 1 06) for per- 20 
forming an image improvement operation to im- 
prove the quality of said image using the image 
correcting information to which the identifying 
information relates. 

25 

16. A computer program product for improving the qual- 
ity of an image produced by a camera module in an 
electronic peripheral device, which is capable of 
processing said image, characterized in that the 
computer program product comprises: 30 

computer program means for making said pe- 
ripheral device store the image correcting infor- 
mation related to said camera module in the 
memory of said peripheral device, 35 



computer program means for making said pe- 
ripheral device receive the image produced by 
said camera module and the identifying infor- 
mation related to said image correcting infor- 40 
mation, 



computer program means for making said pe- 
ripheral device perform image improvement to 
improve the quality of said image using the im- 4$ 
age correcting information to which said re- 
ceived identifying information relates. 
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